A Reliability Generalization Meta-analysis of the Dimensional Obsessive-Compulsive Scale.

BACKGROUND: The Dimensional Obsessive-Compulsive Scale (DOCS) is a well-established tool for assessing obsessive-compulsive symptomatology. A reliability generalization meta-analysis was conducted to estimate the average reliability of DOCS scores and how reliability estimates vary according to the composition and variability of samples, to identify study characteristics that can explain its variability, and to estimate the reliability induction rate. METHOD: A literature search produced 86 studies that met the inclusion criteria. RESULTS: For the DOCS total scores, an average alpha coefficient of .925 was found (95% CI [.920,.931]), as well as mean alphas of .881, .905, .913, and .914 for Contamination, Responsibility, Unacceptable Thoughts, and Symmetry subscales, respectively. Moderator analysis showed that internal consistency fell significantly the more clinical and subclinical participants there were in the sample, and the larger the mean score in the sample for the total scores. The most important moderator variables for the subscales were the standard deviation and the mean of the scores. CONCLUSIONS: The DOCS scores exhibited excellent internal consistency reliability for both total score and subscale scores and DOCS is suitable both for research and clinical purposes.

Improving the reporting quality of reliability generalization meta-analyses: The REGEMA checklist.

Reliability generalization (RG) is a meta-analytic approach that aims to characterize how reliability estimates from the same test vary across different applications of the instrument. With this purpose RG meta-analyses typically focus on a particular test and intend to obtain an overall reliability of test scores and to investigate how the composition and variability of the samples affect reliability. Although several guidelines have been proposed in the meta-analytic literature to help authors improve the reporting quality of meta-analyses, none of them were devised for RG meta-analyses. The purpose of this investigation was to develop REGEMA (REliability GEneralization Meta-Analysis), a 30-item checklist (plus a flow chart) adapted to the specific issues that the reporting of an RG meta-analysis must take into account. Based on previous checklists and guidelines proposed in the meta-analytic arena, a first version was elaborated by applying the nominal group methodology. The resulting instrument was submitted to a list of independent meta-analysis experts and, after discussion, the final version of the REGEMA checklist was reached. In a pilot study, four pairs of coders applied REGEMA to a random sample of 40 RG meta-analyses in Psychology, and results showed satisfactory inter-coder reliability. REGEMA can be used by: (a) meta-analysts conducting or reporting an RG meta-analysis and aiming to improve its reporting quality; (b) consumers of RG meta-analyses who want to make informed critical appraisals of their reporting quality, and (c) reviewers and editors of journals who are considering submissions where an RG meta-analysis was reported for potential publication.